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METHOD OF CODING VIDEO STREAMS FOR 
LOW-COST MULTIPLE DESCRIPTION AT GATEWAYS 

The present invention relates to video coding, and more particularly an improved 
system for splitting and combining multiple description video streams. 

With the advent of digital networks such as the Internet, there has been a demand for 
the ability to provide multimedia communication in real time over such networks. However, 
such multimedia communications, compared to analog communication systems, have been 
hampered by the limited bandwidth provided by the digital networks. To adapt multimedia 
communications to such hardware environments, much effort has been made to develop 
video compression techniques that improve multimedia throughput under limited bandwidth 
conditions using predictive coded video streams. These efforts have led to the emergence of 
several intemational standards such as the MPEG-2 and MPEG-4 standards issued by the 
Motion Pictures Experts Group (MPEG) of the ISO and the H.26L and H.263 standards 
issued by the Video Coding Experts Group (VCEG) of the ITU. These standards achieve a 
high compression ratio by exploiting temporal and spatial correlations in real image 
sequences, using motion-compensated prediction and transform coding. 

More recently diversity techniques, using Multiple Description Coding (MDC), have 
been employed to increase the robustness of communication systems and storage devices. 
Examples of such systems enhanced by diversity techniques include packet networks, 
wireless systems using multi-path and Doppler diversity and Redundant Arrays of 
Inexpensive Disks (RAIDs). 

Present diversity techniques using MDC have worked best in systems were the 
diversity issues are known at the source of the communication. In such instances MDC is 
used to break the data to be communicated into separate pathways each being separately 
coded by the source. One such form of MDC is based on splitting (Fig. 1) a video stream 10 
at a gateway 12, for example, the odd-frames 14 into one description that is coded 
independently with MPEG, or the like, and the even-frames 16 into another description that is 
also coded independently with MPEG, or the like. Each of these streams is then transmitted 
and recombined at the destination. By implementing such methods, it will be appreciated that 
even if one stream is lost the data stream can be performed although at a reduced quality 
level. 
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Now with changes in the way information is delivered between wireless platforms and 
high-speed digital connections, the need for implementing diversity techniques at 
intermediate points in communication pathways is increasing in demand. By increasing the 
ways that hardware pathways are configured, a need has arisen for greater management of 
5 large multimedia data during communication. Presently, gateways that operate to channel 

high bandwidth channels between a plurality of low bandwidth stations have applied diversity 
techniques using MDC by transcoding all of the data. However, such solutions increase the 
overhead experienced at the gateway and may cause an increase in the transmission time. 
Both of these traits are undesirable. Thus, a need exists for a way to increase the advantages 
10 of diversity techniques during transmission, while minimizing the overhead imposed upon 

communication hardware. 

The present invention utilizes a data relationship between B-frame motion vectors and 
P-frame motion vectors to simplify merging and dividing of multiple descriptions at gateways 
by avoiding the need to decompress and re-compress at least one of the multiple descriptions. 
15 One aspect of the invention includes a data stream in which motion vectors of 

succeeding frames correspond to motion vectors of neighboring frames. 

In one embodiment a gateway intermediate in the transmission of a data stream 
utilizes a method of managing multiple descriptions using the motion vector relationships to 
generate or merge multiple descriptions. 
20 Other objects and advantages of the invention will become apparent from the 

foregoing detailed description taken in connection with the accompanying drawings, in which 

FIG. 1 is a block diagram of a known multiple description technique; 

FIG. 2 is a block diagram of a communication pathway; 

FIG. 3 is a block diagram of video frames in a predictive video stream; 
25 FIG. 4 is a block diagram of a multiple-description technique according to the present 

invention; 

FIG. 5 is a block diagram of another multiple-description technique according to the 
present invention; and 

FIG. 6 is a block diagram of a wireless gateway. 
30 With reference to the figures for purposes of illustration, the present invention relates 

to a system for implementing multi-channel transmission in a communications pathway of 
predictive scalable coding schemes. The present invention is presently described in 
connection with a communication system (Fig. 2) including a communication padiway 20 in 
which a communication channel includes multiple transmission pathways 22 and 24 that 
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merge with a single transmission pathway 26 at a gateway 28 or other similar device for 
managing traffic where the pathways merge. It will be appreciated by those skilled in the art 
that this description is merely exemplary of the hardware environment in which this invention 
may be used and that the present invention may be implemented in other hardware 
5 environments as well. Advantageously, the present invention utilizes a mechanism that 

allows for a stream of multimedia data to be split into multiple descriptions without the 
overhead of full transcoding of the data in the stream. 

The invention is implemented upon the realization that a stream of multimedia data 
compressed using predictive coding may be split into multiple descriptions for multiple 

10 transmission pathways without the need to decompress and re-compress the data for multiple 

pathways. Predictive coding techniques of the type suitable for this purpose include MPEG 
standards MPEG-1, MPEG-2 and MPEG-4 as well as ITT standards H.261, H.262, H.263 
and H.26L. With reference to the MPEG standard description for purposes of illustration, a 
movie or video data stream is made up of a sequence of frames that when displayed in 

1 5 sequential order produce the visual effect of animation. Predictive coding produces 

reductions in the amount of data to be transmitted by only transmitting information that 
relates to differences between each sequential frame. Under the MPEG standard, predictive 
coding of frames is based off of an I-frame (Intra-coded frame) that contains all the 
information to 're-build' a frame of video. It should be noted that I-firame only encoded video 

20 does not utilize predictive coding techniques as every fi*ame of the file is independent and 

requires no other frame information. Predictive coding permits greater compression factors 
by removing the redundancy from one frame to the next, in other words sending a set of 
instructions to create the next frame from the current. Such frames are called P-frames 
(Predicted fi-ames). However, a drawback in using I- and P-frame predictive encoding is that 

25 data can only be taken from the previous picture. Moving objects can reveal a background 

that is unknown in previous pictures, while it may be visible in later pictures. B-frames (Bi- 
directional frames) can be created from preceding and/ or later I or P-frames. An I-fi-ame 
with a series of successive B- and P-frames, up to the next I-frame is called a GOP (Group of 
Pictures). An example of a GOP for broadcasting has the structure IBBPBBPBBPBB and is 

30 referred to as IPB-GOP. 

One method of sending multimedia data through two or more pathways uses Multiple 
Description Coding (MDC). MDC has been shown to be an effective technique for robust 
communication over wireless systems using multi-path and Doppler diversity and Redundant 
Arrays of Inexpensive Disks (RAIDs), and also over the Internet. Currently, if an MPEG or 
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H. 26L coded or any other predictive coded video stream of data is transmitted through the 
Internet and then at the gateway it needs to be split into 2 multiple description video streams 
that better fit the channel characteristics of the down-link (e.g. wireless systems using multi- 
path) while preserving the same coding format as before, the video data is fully decoded and 

5 re-encoded. However, the present invention covers a system that allows the gateway to easily 

split a data stream into multiple descriptions without expensive full transcoding while still 
allowing for more resilient transmission. As will be described below this savings in time and 
format is accomplished by coding the hierarchy of motion vectors in a particular format. The 
particular coding format is based on the observation that the motion-vectors for the B-frames 
10 are not very different from part of the motion-vectors (MVs) used for P-frames. 

Normally, independent MVs are computed for B-frames. However (Fig. 3), good 
approximations or predictions for the B-frames' 30 MVs 32 can be computed from the P- 
frames' 34 MVs 36 as Kb(B) and Kf(B) depicted in Figure 2 from the following formula: 

k /^^=-(i-J_) k ; d = k /^^ - k 

M+1 

1 5 where M is the number of B-frames between two consecutive P-frames. 

Thus, the B-frames' MVs could be computed from P-frame MVs and conversely. This coding 
format of the motion vectors is not preferred in current standardized video coding schemes, 
but can be implemented with no change in the standards. However, it shows that more 
accurate motion trajectories can be predicted from sub-sampled trajectories available, i.e. the 

20 B-frames' MVs scan be predicted from the P-frames' MVs. 

Examples: 

I . Splitting A Data Stream Into Two Pathways 

With reference to Fig. 4, the video data is transmitted from the server through a data 
25 channel, for example, but not by way of limitation, through the Internet. The video data, 

transmitted as a single predictive stream 40, then encounters a node 41 along the data channel 
such as a proxy or gateway. For purposes of this application the terms node, gateway and 
proxy may be used interchangeably. At the proxy, the stream is split into 2 separate 
descriptions 42 and 44. To eliminate the complexity associated with full re-encoding of the 
30 streams at the proxy, the video stream transmitted through the channel 40 is coded using an 

IPB GOP-structure, while the two descriptions 42 and 44 transmitted over the wireless link 
use IP GOP-structures. It will be appreciated by those skilled in the art that due to these 
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restrictions, the performance of the coding scheme is reduced. Nevertheless, in this way, one 
MD 42 needs no re-coding at all, while for the other MD 44, the motion estimation at the 
proxy is no longer necessary, since the MVs for the MDs can use H,^^^ and the of the 1^^^^ next 
frame to determine the MVs between P-frames or I and P-frames. Thus, the transition 
5 between a single channel 40 to two descriptions 42 and 44 can be performed easily by re- 

coding only the texture data. All macroblocks without MVs can be coded as intra-blocks. 
Also, if the proxy allows higher complexity processing, further refinements "d" of these 
estimations can be computed. For instance, a new lower complexity motion estimation can 

A 

be performed but using a small search window (e.g. 8 by 8 pixels) centered at kf**^ to find a 
10 more accurate motion vector that would lead to a lower residual (e.g. Maximum Absolute 

Difference) for the newly created P-frame. The computation of the MVs and refinements "d" 
can be derived from the relationship decribed above as follows: 

assuming that in this example there was only 1 B-frame in the initial bitstream between two 
15 consecutive P-frames. Note also that this is just an example and analogous equations can be 

derived if a different number of B-frames are present between 2 consecutive P-frames. 
In an alternate embodiment, the refinements "d" can be computed at the server and sent in a 
separate stream through the Internet. 

20 2. Merging A Data Stream From Two Pathways 

With reference to FIG. 5, if the video stream is received by a proxy 50 over the 
Intemet using two MDs 51 and 52 and the data is further transmitted wirelessly as a single 
stream 54, tfie reverse operation takes place. The MVs for the B-frames can be estimated 

A A A A 

initially as Iq^^ and So initially, 1^ = kf and H> = kb. Then, if the proxy allows higher 
25 complexity processing, further refinements "d" of these estimations can be computed. For 

instance, a new lower complexity motion estimation can be performed but using a small 

A A 

search window (e.g. 8 by 8 pixels) centered at 1^^^^ and H>^®^ to find a more accurate motion 
vector that would lead to a lower residual (e.g. Maximum Absolute Difference) for the newly 
created B-frame. In this case, only the texture coding of the B-frames needs to be re-coded. 
30 The computation of the MVs and refinements "d" use the same relationships as set forth 

above as follows: 
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k /^^=-(.-_L)k^''> ; d = k -k 

M+1 

where M is the number of newly created B-frames between two consecutive available P- 
frames. Note also that this is just an example and analogous equations can be derived if a 
different number of B-frames are created between 2 consecutive P-frames. In an altemate 
5 embodiment, the refinements "d" can be computed at the server and sent in a separate stream 

through the Internet together with the second MD. 

It will be appreciated by those skilled in the art that the proposed method can be 
employed for any predictive coding scheme using Motion-estimation, such as MPEG-1, 2, 4 
and H.263, H.26L. 

10 It will further be appreciated by those skilled in the art that another advantage of this 

method resides in the fact that error recovery and concealment can be performed easier. This 
is because the redundant description of the MVs can be used to determined the MVs for the 
lost frame. 

Finally those skilled in the art will appreciate that this method can be employed for 

15 robust, multi-channel transmission of "predictive" scalable coding schemes, such as Fine 

Granularity Scalable (FGS). This method can be used without MPEG-4 standard 
modifications and thus can be easily employed. 
Uses in Gateway processing: 

With reference to FIG. 6, the present invention has application in gateway 

20 configurations in order to cope with the various network and device characteristics in the 

down-link. The gateway can be located in the home, i.e. a residential gateway, in the 3G 
network, i.e. a base-station or the processing can be distributed across multiple gateways/ 
nodes. In such instances the gateway 60 coimects a Local Area Network (LAN) 62 to the 
Internet 64. As shown in Figure 6, a web server 65 or the like may be enabled to 

25 communicate with local devices 66-68. In instances where the LAN 62 is a wireless down- 

link, devices may include, but are not limited to, mobile PCs 66, Cellular Telephones 67 or 
Portable Data Assistants (PDAs) 68. In such instances the web server 65 and down-link 
devices 66-68 are both unaware of the communication pathways that the data travels. A 
stream of video, when transmitted between the devices, may require dynamic configurations 

30 in which for example the mobile PCs may demand multiple data channels to increase 

bandwidth to the gateway. Or the communication between the gateway and the web server 
may communicate through multiple data channels. In each instance it will be appreciated that 
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the gateway serves to break up the data transmission to service the either the down-link or up- 
link node. The present invention as described in examples 1 and 2 above may be 
implemented in each of these instance to provide a seamless transition at the gateway 
between the up-link and down-link nodes regardless of the number of data channels used. 
5 Currently, if an MPEG or H.26Lcoded or any other predictive coded video stream is 

transmitted through the Internet and then at the gateway it needs to be split into 2 multiple 
descriptions video streams that better fit the channel characteristics of the down-link (e.g. 
wireless systems using multi-path) while preserving the same coding format as before, the 
video data is fully decoded and re-encoded. 

10 By implementing the present invention as described above in which a relationship is 

established between the B -frames' MVs and P-frames' MVs, the present process allows at the 
gateway easy splitting of an MPEG or H.26L coded data or any other predictive coded video 
stream into two multiple descriptions video streams that preserve the same coding format as 
before or results in merging of two multiple descriptions MPEG or H.26L coded or any other 

1 5 predictive coded video streams into a single coded format that preserves the same coding 

format as before without full decoding and re-encoding of the stream. It will be appreciated 
that with the proposed mechanism a considerable amount of the computational complexity at 
the gateway can be reduced. 

While the present invention has been described in connection with what are presently 

20 considered to be the most practical and preferred embodiments, it is to be understood that the 

invention is not to be limited to the disclosed embodiments, but to the contrary, is intended to 
cover various modifications and equivalent arrangements included within the spirit of the 
invention, which are set forth in the appended claims, and which scope is to be accorded the 
broadest interpretation so as to encompass all such modifications and equivalent structures. 
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